Modeling gender differences in the job promotion process: Replication and extension of Martell et al. (1996).

Differences in employee evaluations due to gender bias may be small in any given rating cycle, but they may accumulate to produce large disparities in the number of women and men promoted to the top of an organization. A highly cited simulation by Martell et al. (1996) demonstrates this cumulative advantage process in a multilevel organization. We replicated this simulation to uncover important details about its operating assumptions, and we extended the simulation to examine a range of variables that may impact the cumulative effects of gender bias. The replication revealed that the male cumulative advantage in the Martell et al. simulation requires (a) decades of typical promotion cycles to produce, (b) constant mean differences in the performance ratings of women and men but equal within-group variances, and (c) attrition that occurs randomly at a low and constant rate. Our extended simulation demonstrates that (a) cumulative effects of gender bias are higher when the attrition rate is lower, (b) gender biases are mitigated when attrition is more strongly associated with good or poor performance, and (c) the cumulative effects of mean gender differences in performance ratings can often be smaller than the cumulative effects of variance differences between gender subgroups. Results suggest that talent development and recognition of high performers might have a greater positive impact on female representation at top levels of a firm than programs aimed at reducing bias in employee evaluations. We encourage additional simulation work that further explores the dynamics of cumulative advantage in employment settings. (PsycInfo Database Record (c) 2024 APA, all rights reserved).

The validity of general cognitive ability predicting job-specific performance is stable across different levels of job experience.

Decades of research in industrial-organizational psychology have established that measures of general cognitive ability (g) consistently and positively predict job-specific performance to a statistically and practically significant degree across jobs. But is the validity of g stable across different levels of job experience? The present study addresses this question using historical large-scale data across 31 diverse military occupations from the Joint-Service Job Performance Measurement/Enlistment Standards Project (N = 10,088). Across all jobs, results of our meta-analysis find near-zero interactions between Armed Forces Qualification Test score (a composite of math and verbal scores) and time in service when predicting job-specific performance. This finding supports the validity of g for predicting job-specific performance even with increasing job experience and provides no evidence for diminishing validity of g. We discuss the theoretical and practical implications of these findings, along with directions for personnel selection research and practice. (PsycInfo Database Record (c) 2024 APA, all rights reserved).

What can we learn from COVID-19?: examining the resilience of primary care teams.

Introduction: The COVID-19 pandemic continues to place an unprecedented strain on the US healthcare system, and primary care is no exception. Primary care services have shifted toward a team-based approach for delivering care in the last decade. COVID-19 placed extraordinary stress on primary care teams at the forefront of the pandemic response efforts. The current work applies the science of effective teams to examine the impact of COVID-19-a crisis or adverse event-on primary care team resilience. Methods: Little empirical research has been done testing the theory of team resilience during an extremely adverse crisis event in an applied team setting. Therefore, we conducted an archival study by using large-scale national data from the Veterans Health Administration to understand the characteristics and performance of 7,023 Patient Aligned Care Teams (PACTs) during COVID-19. Results: Our study found that primary care teams maintained performance in the presence of adversity, indicating possible team resilience. Further, team coordination positively predicted team performance (B = 0.53) regardless of the level of adversity a team was experiencing. Discussion: These findings in turn attest to the need to preserve team coordination in the presence of adversity. Results carry implications for creating opportunities for teams to learn and adjust to an adverse event to maintain performance and optimize team-member well-being. Teamwork can act as a protective factor against high levels of workload, burnout, and turnover, and should be studied further for its role in promoting team resilience.

Toward user-centered assistive technologies for aging in place with cognitive impairment: a survey.

PURPOSE: To reduce the burden of Alzheimer's disease, the use of assistive technologies for patients and their informal caregivers is considered essential. However, these technologies are made as "one size fits all" instead of being tailored to accommodate people with varying degrees of cognitive impairment and those with diverse races/ethnicities. Thus, the aim of this survey was to determine whether the types of assistance needed most, and the technology used by those with cognitive impairment differed by race (White/non-Hispanics, Black or African Americans, and Hispanic/Latinos or Puerto Ricans) and severity of dementia (mild, moderate, severe). RESEARCH DESIGN AND METHODS: One hundred and eighty informal caregivers of people with different levels of severity of cognitive impairment and several different races/ethnicities filled out an online survey regarding assistance needed and technologies used. RESULTS: The results show that racial minorities considered the needs for assistance with Basic Activities of Daily Living as more important compared to White/non-Hispanics with mild dementia. Furthermore, Hispanic/Latinos or Puerto Ricans and White/non-Hispanics with severe dementia were shown to use technology that is designed to help with Instrumental Activities of Daily Living more than those with moderate dementia. Lastly, during COVID-19, devices to assist with walking, preparing meals and personal hygiene have been used significantly more by White/non-Hispanics with severe dementia compared to Hispanic/Latinos or Puerto Ricans. CONCLUSION: The results point to the need to design for those with severe dementia, regardless of race, and should focus on addressing needs related to both Instrumental and Basic Activities of Daily Living.

Developers of assistive technology should consider designing technology that can accommodate all severity levels of cognitive impairment.More research is needed to determine the usability of assistive technology that is designed for those with cognitive impairments.

Understanding the Leaders of Tomorrow: The Need to Study Leadership in Adolescence.

Leadership traits and behaviors are observed early in human development, and although an improved understanding of youth leadership would usefully inform many real-world contexts (e.g., education, parenting, policy), most empirical work on leadership has been limited to adult populations. The purpose of the current article is to add a developmental perspective to leadership research that has so far been absent. Here, we (a) highlight adolescence as a critical developmental period for leadership emergence and development, (b) argue that leadership among youths is poorly understood and critically understudied, (c) provide exemplars of synergy between research on leadership and adolescent development that are ripe for focused inquiry, and (d) underscore some of the positive consequences of accelerating empirical research on leadership in adolescence, including implications for a deeper understanding of leadership in adult working populations.

An evidence-based, structured, expert approach to selecting essential indicators of primary care quality.

BACKGROUND: The purpose of this article is to illustrate the application of an evidence-based, structured performance measurement methodology to identify, prioritize, and (when appropriate) generate new measures of health care quality, using primary care as a case example. Primary health care is central to the health care system and health of the American public; thus, ensuring high quality is essential. Due to its complexity, ensuring high-quality primary care requires measurement frameworks that can assess the quality of the infrastructure, workforce configurations, and processes available. This paper describes the use of the Productivity Measurement and Enhancement System (ProMES) to compile a targeted set of such measures, prioritized according to their contribution and value to primary care. METHODS: We adapted ProMES to select and rank existing primary care measures according to value to the primary care clinic. Nine subject matter experts (SMEs) consisting of clinicians, hospital leaders and national policymakers participated in facilitated expert elicitation sessions to identify objectives of performance, corresponding measures, and priority rankings. RESULTS: The SMEs identified three fundamental objectives: access, patient-health care team partnerships, and technical quality. The SMEs also selected sixteen performance indicators from the 44 pre-vetted, currently existing measures from three different data sources for primary care. One indicator, Team 2-Day Post Discharge Contact Ratio, was selected as an indicator of both team partnerships and technical quality. Indicators were prioritized according to value using the contingency functions developed by the SMEs. CONCLUSION: Our article provides an actionable guide to applying ProMES, which can be adapted to the needs of various industries, including measure selection and modification from existing data sources, and proposing new measures. Future work should address both logistical considerations (e.g., data capture, common data/programming language) and lingering measurement challenges, such as operationalizating measures to be meaningful and interpretable across health care settings.

A template for preregistration of quantitative research in psychology: Report of the joint psychological societies preregistration task force.

Recent years have seen dramatic changes in research practices in psychological science. In particular, preregistration of study plans before conducting a study has been identified as an important tool to help increase the transparency of science and to improve the robustness of psychological research findings. This article presents the Psychological Research Preregistration-Quantitative (PRP-QUANT) Template produced by a Joint Psychological Societies Preregistration Task Force consisting of the American Psychological Association (APA), the British Psychological Society (BPS), and the German Psychological Society (DGPs), supported by the Center for Open Science (COS) and the Leibniz Institute for Psychology (ZPID). The goal of the Task Force was to provide the psychological community with a consensus template for the preregistration of quantitative research in psychology, one with wide coverage and the ability, if necessary, to adapt to specific journals, disciplines, and researcher needs. This article covers the structure and use of the PRP-QUANT template, while outlining and discussing the benefits of its use for researchers, authors, funders, and other relevant stakeholders. We hope that by introducing this template and by demonstrating the support of preregistration by major academic psychological societies, we will facilitate an increase in preregistration practices and also the further advancement of transparency and knowledge-sharing in the psychological sciences. (PsycInfo Database Record (c) 2022 APA, all rights reserved).

A Meta-analysis of University STEM Summer Bridge Program Effectiveness.

University science, technology, engineering, and math (STEM) summer bridge programs provide incoming STEM university students additional course work and preparation before they begin their studies. These programs are designed to reduce attrition and increase the diversity of students pursuing STEM majors and STEM career paths. A meta-analysis of 16 STEM summer bridge programs was conducted. Results showed that program participation had a medium-sized effect on first-year overall grade point average (d = 0.34) and first-year university retention (Odds Ratio [OR] = 1.747). Although this meta-analytic research reflects a limited amount of available quantitative academic data on summer STEM bridge programs, this study nonetheless provides important quantitative inroads into much-needed research on programs' objective effectiveness. These results articulate the importance of thoughtful experimental design and how further research might guide STEM bridge program development to increase the success and retention of matriculating STEM students.

Is It Time to Go Positive? Assessing the Positively Worded System Usability Scale (SUS).

OBJECTIVE: The goal of the research presented in this paper was to determine if the positively worded System Usability Scale (SUS) can be used in place of the positively and negatively worded standard SUS instrument for the subjective assessment of usability, and whether the results found here replicate those of Sauro and Lewis. BACKGROUND: Sauro and Lewis' previous study found no evidence that responses to SUS items differed across the standard SUS and the modified, positively worded version of the SUS when participants assessed websites. This study replicates and extends this work by examining a large number of different systems with larger sample sizes to add to the generalizability of previous findings. METHODS: So that participants could retrospectively assess 20 products, the standard SUS was administered to 268 participants and the positive SUS to 698 participants. SUS scores were computed and the data analyzed using psychometric methods to explore how the two versions of the SUS differed. RESULTS: The standard and positive versions of the SUS yielded similar SUS scores. In addition, both versions of the scale demonstrated evidence in support of reliability and validity. CONCLUSION: Either version of the SUS can be used with confidence to measure subjective usability. Furthermore, the scores generated from both versions of the SUS can be directly compared. APPLICATIONS: In situations where cognitive load, participants' spoken language, or item consistency with other surveys being given may be a factor, the positive SUS is a viable alternative to the standard SUS.

Is the Deliberate Practice View Defensible? A Review of Evidence and Discussion of Issues.

The question of what explains individual differences in expertise within complex domains such as music, games, sports, science, and medicine is currently a major topic of interest in a diverse range of fields, including psychology, education, and sports science, to name just a few. Ericsson and colleagues' deliberate practice view is a highly influential perspective in the literature on expertise and expert performance-but is it viable as a testable scientific theory? Here, reviewing more than 25 years of Ericsson and colleagues' writings, we document critical inconsistencies in the definition of deliberate practice, along with apparent shifts in the standard for evidence concerning deliberate practice. We also consider the impact of these issues on progress in the field of expertise, focusing on the empirical testability and falsifiability of the deliberate practice view. We then discuss a multifactorial perspective on expertise, and how open science practices can accelerate progress in research guided by this perspective.

The Practical Effects of Measurement Invariance: Gender Invariance in Two Big Five Personality Measures.

The essence of measurement invariance (MI) analysis is to test the assumption that observed scores on a scale accurately reflect respondents' standings on a measured construct. Based on exploratory structural equation modeling, the current study examines gender-based MI in two Big Five measures of personality: the Mini-IPIP, and the Big Five Inventory (BFI) facet scales. We report results for MI based on both model fit indices and a practical significance index that quantifies the extent of noninvariance (i.e., dMACS). From the latter, we partition the observed group mean differences in scale scores into construct-irrelevant group differences versus construct-relevant group differences. In measures of Agreeableness, Conscientiousness, Extraversion, and Neuroticism across instruments, results supported metric invariance but not scalar invariance. That said, findings of statistical noninvariance were generally small in terms of practical effects, although some notable variability in the effects was evident. Overall, the current results provide evidence regarding gender-based MI of Big Five personality measures that are more detailed than that provided in past work. More generally, this study also provides useful guidance for future researchers investigating both the statistical and practical significance of measurement invariance.

Modeling Measurement as a Sequential Process: Autoregressive Confirmatory Factor Analysis (AR-CFA).

To model data from multi-item scales, many researchers default to a confirmatory factor analysis (CFA) approach that restricts cross-loadings and residual correlations to zero. This often leads to problems of measurement-model misfit while also ignoring theoretically relevant alternatives. Existing research mostly offers solutions by relaxing assumptions about cross-loadings and allowing residual correlations. However, such approaches are critiqued as being weak on theory and/or indicative of problematic measurement scales. We offer a theoretically-grounded alternative to modeling survey data called an autoregressive confirmatory factor analysis (AR-CFA), which is motivated by recognizing that responding to survey items is a sequential process that may create temporal dependencies among scale items. We compare an AR-CFA to other common approaches using a sample of 8,569 people measured along five common personality factors, showing how the AR-CFA can improve model fit and offer evidence of increased construct validity. We then introduce methods for testing AR-CFA hypotheses, including cross-level moderation effects using latent interactions among stable factors and time-varying residuals. We recommend considering the AR-CFA as a useful complement to other existing approaches and treat AR-CFA limitations.

Impact of team configuration and team stability on primary care quality.

BACKGROUND: The science of effective teams is well documented; far less is known, however, about how specific team configurations may impact primary care team effectiveness. Further, teams experiencing frequent personnel changes (perhaps as a consequence of poor implementation) may have difficulty delivering effective, continuous, well-coordinated care. This study aims to examine the extent to which primary care clinics in the Veterans Health Administration have implemented team configurations consistent with recommendations based on the Patient-Centered Medical Home model and the extent to which adherence to said recommendations, team stability, and role stability impact healthcare quality. Specifically, we expect to find better clinical outcomes in teams that adhere to recommended team configurations, teams whose membership and configuration are more stable over time, and teams whose clinical manager role is more stable over time. METHODS/DESIGN: We will employ a combination of social network analysis and multilevel modeling to conduct a database review of variables extracted from the Veterans Health Administration's Team Assignments Report (TAR) (one of the largest, most diverse existing national samples of primary care teams (nteams > 7000)), as well as other employee and clinical data sources. To ensure the examination of appropriate clinical outcomes, we will enlist a team of subject matter experts to select a concise set of clear, prioritized primary care performance metrics. We will accomplish this using the Productivity Measurement and Enhancement System, an evidence-based methodology for developing and implementing performance measurement. DISCUSSION: We are unaware of other studies of healthcare teams that consider team size, composition, and configuration longitudinally or with sample sizes of this magnitude. Results from this study can inform primary care team implementation policy and practice in both private- and public-sector clinics, such that teams are configured optimally, with adequate staffing, and the right mix of roles within the team. TRIAL REGISTRATION: Not applicable-this study does not involve interventions on human participants.

Applying Task Force Recommendations on Integrating Science and Practice in Health Service Psychology Education.

The proper role of research skills and training to conduct research in professional psychology education has been controversial throughout the history of the field. An extensive effort was undertaken recently to address that issue and identify ways the field might move forward in a more unified manner. In 2015, the American Psychological Association (APA) Board of Educational Affairs convened a task force to address one of the recommendations made by the Health Service Psychology Education Collaborative in 2013. That recommendation stated that the education and training of health service psychologists (HSPs) include an integrative approach to science and practice that incorporates scientific-mindedness, training in research skills, and goes well beyond merely "consuming" research findings. The task force subsequently developed recommendations related to the centrality of science competencies for HSPs and how these competencies extend beyond training in evidence-based practice. This article discusses the findings of the task force and the implications of its recommendations for education and training in HSP. The challenges and opportunities associated with implementing these recommendations in HSP graduate programs are examined.

The accuracy of dominance analysis as a metric to assess relative importance: The joint impact of sampling error variance and measurement unreliability.

Dominance analysis (DA) has been established as a useful tool for practitioners and researchers to identify the relative importance of predictors in a linear regression. This article examines the joint impact of two common and pervasive artifacts-sampling error variance and measurement unreliability-on the accuracy of DA. We present Monte Carlo simulations that detail the decrease in the accuracy of DA in the presence of these artifacts, highlighting the practical extent of the inferential mistakes that can be made. Then, we detail and provide a user-friendly program in R (R Core Team, 2017) for estimating the effects of sampling error variance and unreliability on DA. Finally, by way of a detailed example, we provide specific recommendations for how researchers and practitioners should more appropriately interpret and report results of DA. (PsycINFO Database Record (c) 2019 APA, all rights reserved).

Big data in psychology: Introduction to the special issue.

The introduction to this special issue on psychological research involving big data summarizes the highlights of 10 articles that address a number of important and inspiring perspectives, issues, and applications. Four common themes that emerge in the articles with respect to psychological research conducted in the area of big data are mentioned, including: (a) The benefits of collaboration across disciplines, such as those in the social sciences, applied statistics, and computer science. Doing so assists in grounding big data research in sound theory and practice, as well as in affording effective data retrieval and analysis. (b) Availability of large data sets on Facebook, Twitter, and other social media sites that provide a psychological window into the attitudes and behaviors of a broad spectrum of the population. (c) Identifying, addressing, and being sensitive to ethical considerations when analyzing large data sets gained from public or private sources. (d) The unavoidable necessity of validating predictive models in big data by applying a model developed on 1 dataset to a separate set of data or hold-out sample. Translational abstracts that summarize the articles in very clear and understandable terms are included in Appendix A, and a glossary of terms relevant to big data research discussed in the articles is presented in Appendix B. (PsycINFO Database Record

Using the IAT to predict ethnic and racial discrimination: small effect sizes of unknown societal significance.

Greenwald, Banaji, and Nosek (2015) present a reanalysis of the meta-analysis by Oswald, Mitchell, Blanton, Jaccard, and Tetlock (2013) that examined the effect sizes of Implicit Association Tests (IATs) designed to predict racial and ethnic discrimination. We discuss points of agreement and disagreement with respect to methods used to synthesize the IAT studies, and we correct an error by Greenwald et al. that obscures a key contribution of our meta-analysis. In the end, all of the meta-analyses converge on the conclusion that, across diverse methods of coding and analyzing the data, IAT scores are not good predictors of ethnic or racial discrimination, and explain, at most, small fractions of the variance in discriminatory behavior in controlled laboratory settings. The thought experiments presented by Greenwald et al. go well beyond the lab to claim systematic IAT effects in noisy real-world settings, but these hypothetical exercises depend crucially on untested and, arguably, untenable assumptions.

Comparing Simple Scoring With IRT Scoring of Personality Measures: The Navy Computer Adaptive Personality Scales.

This article analyzes data from U.S. Navy sailors (N = 8,956), with the central measure being the Navy Computer Adaptive Personality Scales (NCAPS). Analyses and results from this article extend and qualify those from previous research efforts by examining the properties of the NCAPS and its adaptive structure in more detail. Specifically, this article examines item exposure rates, the efficiency of item use based on item response theory (IRT)-based Expected A Posteriori (EAP) scoring, and a comparison of IRT-EAP scoring with much more parsimonious scoring methods that appear to work just as well (stem-level scoring and dichotomous scoring). The cutting-edge nature of adaptive personality testing will necessitate a series of future efforts like this: to examine the benefits of adaptive scoring schemes and novel measurement methods continually, while pushing testing technology further ahead.

The development of a short domain-general measure of working memory capacity.

Working memory capacity is one of the most frequently measured individual difference constructs in cognitive psychology and related fields. However, implementation of complex span and other working memory measures is generally time-consuming for administrators and examinees alike. Because researchers often must manage the tension between limited testing time and measuring numerous constructs reliably, a short and effective measure of working memory capacity would often be a major practical benefit in future research efforts. The current study developed a shortened computerized domain-general measure of working memory capacity by representatively sampling items from three existing complex working memory span tasks: operation span, reading span, and symmetry span. Using a large archival data set (Study 1, N = 4,845), we developed and applied a principled strategy for developing the reduced measure, based on testing a series of confirmatory factor analysis models. Adequate fit indices from these models lent support to this strategy. The resulting shortened measure was then administered to a second independent sample (Study 2, N = 172), demonstrating that the new measure saves roughly 15 min (30%) of testing time on average, and even up to 25 min depending on the test-taker. On the basis of these initial promising findings, several directions for future research are discussed.